30 research outputs found
Requirements Analysis of a Quad-Redundant Flight Control System
In this paper we detail our effort to formalize and prove requirements for
the Quad-redundant Flight Control System (QFCS) within NASA's Transport Class
Model (TCM). We use a compositional approach with assume-guarantee contracts
that correspond to the requirements for software components embedded in an AADL
system architecture model. This approach is designed to exploit the
verification effort and artifacts that are already part of typical software
verification processes in the avionics domain. Our approach is supported by an
AADL annex that allows specification of contracts along with a tool, called
AGREE, for performing compositional verification. The goal of this paper is to
show the benefits of a compositional verification approach applied to a
realistic avionics system and to demonstrate the effectiveness of the AGREE
tool in performing this analysis.Comment: Accepted to NASA Formal Methods 201
Input Prioritization for Testing Neural Networks
Deep neural networks (DNNs) are increasingly being adopted for sensing and
control functions in a variety of safety and mission-critical systems such as
self-driving cars, autonomous air vehicles, medical diagnostics, and industrial
robotics. Failures of such systems can lead to loss of life or property, which
necessitates stringent verification and validation for providing high
assurance. Though formal verification approaches are being investigated,
testing remains the primary technique for assessing the dependability of such
systems. Due to the nature of the tasks handled by DNNs, the cost of obtaining
test oracle data---the expected output, a.k.a. label, for a given input---is
high, which significantly impacts the amount and quality of testing that can be
performed. Thus, prioritizing input data for testing DNNs in meaningful ways to
reduce the cost of labeling can go a long way in increasing testing efficacy.
This paper proposes using gauges of the DNN's sentiment derived from the
computation performed by the model, as a means to identify inputs that are
likely to reveal weaknesses. We empirically assessed the efficacy of three such
sentiment measures for prioritization---confidence, uncertainty, and
surprise---and compare their effectiveness in terms of their fault-revealing
capability and retraining effectiveness. The results indicate that sentiment
measures can effectively flag inputs that expose unacceptable DNN behavior. For
MNIST models, the average percentage of inputs correctly flagged ranged from
88% to 94.8%
Formal Methods Case Studies for DO-333
RTCA DO-333, Formal Methods Supplement to DO-178C and DO-278A provides guidance for software developers wishing to use formal methods in the certification of airborne systems and air traffic management systems. The supplement identifies the modifications and additions to DO-178C and DO-278A objectives, activities, and software life cycle data that should be addressed when formal methods are used as part of the software development process. This report presents three case studies describing the use of different classes of formal methods to satisfy certification objectives for a common avionics example - a dual-channel Flight Guidance System. The three case studies illustrate the use of theorem proving, model checking, and abstract interpretation. The material presented is not intended to represent a complete certification effort. Rather, the purpose is to illustrate how formal methods can be used in a realistic avionics software development project, with a focus on the evidence produced that could be used to satisfy the verification objectives found in Section 6 of DO-178C
Towards Realizability Checking of Contracts using Theories
Virtual integration techniques focus on building architectural models of
systems that can be analyzed early in the design cycle to try to lower cost,
reduce risk, and improve quality of complex embedded systems. Given appropriate
architectural descriptions and compositional reasoning rules, these techniques
can be used to prove important safety properties about the architecture prior
to system construction. Such proofs build from "leaf-level" assume/guarantee
component contracts through architectural layers towards top-level safety
properties. The proofs are built upon the premise that each leaf-level
component contract is realizable; i.e., it is possible to construct a component
such that for any input allowed by the contract assumptions, there is some
output value that the component can produce that satisfies the contract
guarantees. Without engineering support it is all too easy to write leaf-level
components that can't be realized. Realizability checking for propositional
contracts has been well-studied for many years, both for component synthesis
and checking correctness of temporal logic requirements. However, checking
realizability for contracts involving infinite theories is still an open
problem. In this paper, we describe a new approach for checking realizability
of contracts involving theories and demonstrate its usefulness on several
examples.Comment: 15 pages, to appear in NASA Formal Methods (NFM) 201
Manifold-based Test Generation for Image Classifiers
Neural networks used for image classification tasks in critical applications
must be tested with sufficient realistic data to assure their correctness. To
effectively test an image classification neural network, one must obtain
realistic test data adequate enough to inspire confidence that differences
between the implicit requirements and the learned model would be exposed. This
raises two challenges: first, an adequate subset of the data points must be
carefully chosen to inspire confidence, and second, the implicit requirements
must be meaningfully extrapolated to data points beyond those in the explicit
training set. This paper proposes a novel framework to address these
challenges. Our approach is based on the premise that patterns in a large input
data space can be effectively captured in a smaller manifold space, from which
similar yet novel test cases---both the input and the label---can be sampled
and generated. A variant of Conditional Variational Autoencoder (CVAE) is used
for capturing this manifold with a generative function, and a search technique
is applied on this manifold space to efficiently find fault-revealing inputs.
Experiments show that this approach enables generation of thousands of
realistic yet fault-revealing test cases efficiently even for well-trained
models
Certification Considerations for Adaptive Systems
Advanced capabilities planned for the next generation of aircraft, including those that will operate within the Next Generation Air Transportation System (NextGen), will necessarily include complex new algorithms and non-traditional software elements. These aircraft will likely incorporate adaptive control algorithms that will provide enhanced safety, autonomy, and robustness during adverse conditions. Unmanned aircraft will operate alongside manned aircraft in the National Airspace (NAS), with intelligent software performing the high-level decision-making functions normally performed by human pilots. Even human-piloted aircraft will necessarily include more autonomy. However, there are serious barriers to the deployment of new capabilities, especially for those based upon software including adaptive control (AC) and artificial intelligence (AI) algorithms. Current civil aviation certification processes are based on the idea that the correct behavior of a system must be completely specified and verified prior to operation. This report by Rockwell Collins and SIFT documents our comprehensive study of the state of the art in intelligent and adaptive algorithms for the civil aviation domain, categorizing the approaches used and identifying gaps and challenges associated with certification of each approach
Pattern-based Composition and Analysis of Virtually Synchronized Real-Time Distributed Systems
Designing and verifying distributed protocols in a multi-rate asynchronous system is, in general, extremely difficult when the distributed computations require consistent input views, consistent actions and synchronized state transitions. In this paper, we address this problem and introduce a formal, complexity-reducing architectural pattern, called Multi-Rate PALS system, to support virtual synchronization in multi-rate distributed computations. The pattern supports a component to be virtually synchronized with other components in different instantiations of this pattern. We present an application of a hierarchical control system to show that the composition of these instantiations can be used to achieve desired system-level properties, such as distributed consistency and distributed coordination. We verify the logical synchronization guarantee of this pattern which holds as long as the pattern assumptions are satisfied. We also discuss the correctness analysis necessary to validate these assumptions and provide a tool support to perform this analysis automatically on the AADL models.unpublishednot peer reviewe
Circular Hierarchical Reasoning using Past Time LTL
Associated research group: Critical Systems Research GroupWe describe a composition rule for
hierarchically composed components that may involve circular
reasoning between the components. It is similar to previous
work by McMillan, specialized to component level reasoning. In
contrast to McMillan's work, our composition rule can be used
in provers that only support safety properties (e.g. k-induction model
checkers) as long as the system and component contracts consist of
state invariants. The composition rule still holds for richer
contracts, but the resulting verification conditions then require a
general purpose model checker